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Abstract 

Ideal  Decision  Support  Systems  (DSS)  provide  aid  in  the  attainment  of  a  solution  to  a  particular 
problem  of  the  user.  However,  during  system  interaction,  the  dialogue  design  of  the  DSS  has  the 
potential  to  influence  the  outcome  of  the  solution.  This  side  effect  may  or  may  not  be  desirable,  but 
system  designers  must  be  aware  of  the  potential  impact.  A  laboratory  study  described  in  this  report 
examines  the  significance  of  the  impact.  A  total  of  46  subjects  conducted  decisions  on  forecasting 
methods  under  two  different  design  structures  for  DSS  interfaces.  An  increase  in  guidance  provided 
by  the  system  led  to  a  significant  change  in  the  decision  model  selected.  This  change  in  model 
selected  resulted  in  a  number  of  different  solutions  to  the  study’s  forecasting  problem.  In  application 
settings,  such  impacts  need  to  be  evaluated  prior  to  implementation  to  avoid  the  situations  where  the 
software  influences  the  decision  process.  ©  2000  Elsevier  Science  B.V.  All  rights  reserved. 
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1.  Introduction 

Interest  in  the  use  of  computer- supported  decision  making  has  grown  in  recent  years. 
Such  systems  are  termed  Decision  Support  Systems  (DSSs).  They  facilitate  decision 
making  by  empowering  users  with  essential  data  and  appropriate  quantitative  models. 
In  the  course  of  interacting  with  a  DSS,  users  may  have  numerous  opportunities  for 
exercising  judgments.  Some  judgments  pertain  to  task  ordering,  others  require  input  of 
predictions  or  evaluations  otherwise  unavailable  to  the  system.  Some  judgements  concern 
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what  to  do  next:  Should  I  run  a  moving- average  or  a  regression  forecasting  method  on  the 
data?  Should  I  run  the  same  data  analysis  again  with  different  parameters?  Other  judge¬ 
ments  are  predictive:  What  will  the  interest  rate  be  in  the  second  quarter  of  this  year?  How 
many  units  of  my  product  will  I  sell  this  year? 

An  interactive  DSS  must  provide  a  means  for  users  to  communicate  these  judgmental 
inputs.  Menus,  fill-in  tables,  and  on-line  help-screens  are  used  widely  for  this  purpose. 
When  interacting  with  a  DSS,  a  key  issue  is  whether  the  dialogue  mechanisms  affect  the 
substance  of  users’  judgmental  inputs.  As  examples,  menus  may  lead  to  order  effects  as 
users  may  tend  to  select  the  first  or  last  items  on  menus  [1].  Use  of  graphics  as  opposed  to 
data  sets  impact  judgment  [2],  For  instance,  numerical  data  are  commonly  organized  into 
tabular  arrangements,  while  there  are  a  variety  of  standard  graphical  arrangements  of 
pictorial  elements  (e.g.  bar  charts,  scatter  plots,  and  so  on).  Some  researchers  have 
found  that  graphical  presentations  are  more  useful  when  evaluating  information  in  order 
to  determine  promising  directions  in  the  search  for  an  optimal  solution  but  when  the  task 
requires  the  determination  of  exact  data  values  for  computational  purposes,  graphical 
reports  are  less  useful  than  tabular  ones  [3].  On  the  other  extreme,  a  DSS  may  embed  a 
knowledge-based  system  that  recommends  actions  and  responses  to  its  users.  The  purpose 
is  to  guide  the  user  toward  a  solution  in  a  rigid  algorithmic  fashion.  Either  deliberately  or 
inadvertently  a  DSS  may  guide  its  users  in  performing  the  required  judgments  [4,5]. 

Recent  DSS  studies  focused  on  important  characteristics  and  design  issues  [6,7].  This 
stream  of  research  found  that  when  a  DSSs  assistance  matches  the  users’  mental  models, 
the  more  easily  and  quickly  users  learn  the  system.  We  should  not  limit  ourselves  to  the 
study  of  problems  concerning  only  existing  technologies  (e.g.  mouse,  voice,  windows,  and 
graphics).  We  should  explore  new,  creative  uses  of  advanced  technologies  to  know  what, 
when,  and  how  to  apply  them  effectively.  We  need  to  broaden  research  concerning  how 
people  organize,  store,  and  retrieve  concepts  [8].  We  also  need  to  investigate  psycholo¬ 
gical  attribute  (such  as  attitude  and  preference),  work-related  factors  (such  as  fatigue  and 
organizational  culture),  and  certain  physical  limitations  (such  as  hearing  and  vision 
impairment).  But  this  cannot  be  easily  achieved  because  existing  technological  restrictions 
limit  the  choice  of  dialog  style  and  impose  rigid  syntax  rules  and  recovery  procedures. 
Further,  the  work  is  hindered  by  our  limited  understanding  of  users’  cognitive  models. 

To  mitigate  these  problems,  [9]  argued  that  a  DSS  should  provide  multi-level  assistance 
to  help  users  learn  and  use  the  system.  For  example,  query-in-depth  is  a  technique 
designed  to  provide  multi-level  assistance  to  help  users  at  various  levels  of  expertise 
learn  the  system  [10].  Its  low-level  help  could  include  brief  ‘how-to-do-it’  and  ‘what-it- 
is-for’  information  that  instructs  users’  immediate  actions.  If  not  satisfied,  the  user  can 
request  more  advanced  ‘how-it-works’  information  for  trouble-shooting.  The  design  ques¬ 
tions  include  what  information  should  be  given  and  when  for  different  tasks,  what  ideas 
should  be  left  to  users  inference,  and  how  to  use  feedback  to  make  learning  enjoyable. 
When  novice  users  make  errors  and  are  uncertain  about  what  to  do  next,  they  often  look  for 
instructions  from  the  system  [11],  Specific  ‘how-to-do-it’  information  can  be  included  to 
assist  novice  users  in  the  quick  completion  of  tasks  [12],  This  complex  interaction  usually 
leads  to  the  requirement  of  prototypes  to  test  and  refine  the  interaction  model.  However, 
such  complex  designs  and  elongated  development  is  expensive  and  time-consuming. 

The  complications  and  costs  described  above  are  why  most  of  implemented  DSSs  lack 
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Fig.  1 .  Evaluation  table  for  forecasting  methods  without  weighting  assistance. 


sophisticated  interactions,  such  as  decisional  guidance  [13].  Decisional  guidance  is  an 
interface  concept  to  assist  a  user  in  the  completion  of  tasks  by  performing  functions 
usually  left  to  the  discretion  of  the  user.  The  decisional  guidance  can  be  either  prede¬ 
fined-informative  or  participative-suggestive.  Predefined-informative  guidance  provides 
certain  information  to  the  user  and  requests  additional  input  to  complete  a  defined  task. 
Participative-suggestive  guidance  segments  the  task  and  provides  guidance  steps  depen¬ 
dent  upon  the  action  previously  taken  by  the  user. 

Recent  literature  proposes  the  use  of  decision  guidance  structures  where  a  decision 
support  system  directs  its  users  as  they  structure  and  execute  their  decision-making 
process.  Predefined-informative  guidance  provides  pertinent  information  that  assists  the 
decision-maker’s  judgement,  without  suggesting  how  to  act.  Participative- suggestive 
guidance  mechanisms  facilitate  users’  on  a  more  detailed  level,  perhaps  even  prompting 
for  needed  information.  For  example,  perhaps  there  is  a  choice  of  forecasting  techniques  in 
a  DSS.  The  DSS  can  support  the  decision  maker’s  meta-choice  by  providing  a  list  of  these 
forecasting  techniques  and  their  associated  properties  (predefined-informative  guidance). 
Kleindorfer  [14]  suggest  that  in  differing  situations,  different  criteria  are  appropriate  for 
justifying  the  choice  of  a  problem-solving  technique.  A  table  such  as  Fig.  1,  which  rates 


Method  Quickness  Expense  Frequency  Consistency  Simplicity  Detail  Weighted 


Quickness:  Can  the  forecasting  model  be  developed  immediately?  From  1  (extremely  slow)  to  10  (extremely  quick) 

Expense:  Can  the  model  be  developed  inexpensively?  From  1  (extremely  expensive)  to  10  (extremely  inexpensive) 

Frequency:  Can  the  model  be  updated  easily  and  rapidly?  From  1  (extremely  difficult)  to  10  (extremely  easy) 

Simplicity:  Can  the  model  be  understood  easily?  From  1  ( extremely  difficult)  to  10  (extremely  easy). 

Scale:  1  (Low  on  the  attribute)  to  10  (High  on  the  attribute) 


Fig.  2.  Evaluation  table  for  forecasting  methods  with  weighting  assistance. 
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the  forecasting-techniques  with  respect  to  selection  criteria,  could  therefore  guide  decision 
makers  by  its  inclusion  of  the  criteria  and  visual  representation  of  the  problem.  By  itself, 
the  table  offers  predefined-informative  guidance.  When  assisting  functions  are  added  that 
empower  users  to  manipulate  the  table  (e.g.  by  allowing  ranks  for  the  criteria  and  comput¬ 
ing  weighted  scores  as  in  Fig.  2),  the  enhanced  table  becomes  a  participative-suggestive 
guidance  mechanism  [13]. 

Previously,  interface  designs,  such  as  the  popular  pull-down  menus  and  help  screens, 
were  oriented  more  toward  the  mechanics  of  operating  system  features  than  toward  the 
decisional  and  judgmental  tasks  at  hand.  However,  a  number  of  recent  researchers  have 
begun  to  develop  decisional  guidance  mechanisms  [15,16].  Silver  [13]  touts  the  value  of 
these  interfaces  which  facilitate  users  deriving  their  own  recommendations.  An  effective 
participative-suggestive  decision  guidance  mechanism  enables  users  to  more  easily 
manipulate  the  logic  (rules)  underlying  the  suggestions. 

For  our  specific  example,  consider  the  first  judgmental  task  often  faced  by  a  DSS  user: 
selecting  an  appropriate  forecasting  model.  A  table,  such  as  Fig.  1  above,  could  serve  to 
guide  decision  makers.  Different  users  would  likely  use  different  selection  strategies  for 
selecting  a  forecasting  technique.  However,  when  the  functions  in  Fig.  2  are  added,  the 
enhanced  table  becomes  a  participative-suggestive  guidance  mechanism.  A  simple 
enhancement  to  the  table  in  Fig.  1,  shown  in  Fig.  2,  adds  a  row  to  allow  the  assignment 
of  weights  to  each  criteria  and  a  column  to  accumulate  weighted  totals  for  each  model.  In 
Fig.  2  the  DSS  designer  provided  basic  information  plus  a  technique  (multi-attributed 
weighting)  to  assist  in  the  selection  of  a  model.  However,  the  user  still  determines  how  the 
multi-attributed  weighting  technique  is  applied,  if  at  all,  in  order  to  select  a  model. 

The  purpose  of  this  study  is  to  examine  the  impact  of  participative-suggestive  guidance 
on  a  user’s  selection  strategy  and  his  decision  making.  Studying  such  deliberate  guidance 
is  important  for  three  reasons.  First,  incorporating  guidance  in  a  DSS  offers  the  potential  of 
more  supportive  systems  while  raising  a  number  of  design  questions.  Second,  understand¬ 
ing  the  consequence  of  deliberate  guidance  contributes  to  comprehending  how  DSSs  may 
affect  decision-making  behavior.  Third,  for  management,  decisional  guidance  represents 
a  set  of  new  management  opportunities  for  improving,  standardizing  and  controlling 
decision  performance. 


2.  Experimental  design 

To  study  the  potential  impact  of  deliberate  guidance  in  a  DSS  on  users’  judgment,  we 
examine  when  users  interact  with  a  model-based  DSS,  how  participative-suggestive 
guidance  using  a  multi-attributed  weighting  technique  influences  users’  choice  of  decision 
strategy  for  selecting  a  forecasting  model.  We  expect  that  subjects  will  change  their 
selection  of  decision  strategies  when  provided  with  participative-suggestive  guidance 
after  having  made  a  selection  with  no  participative-suggestive  guidance. 

Literature  in  social  psychology  suggests  the  effects  of  decisional  guidance  on  strategies 
could  be  analyzed  in  terms  of  the  effort  with  which  various  decision  strategies  can 
be  accomplished  [17,18].  Participative-suggestive  guidance  in  a  DSS  provides  such  a 
cognitive  incentive  for  decision  makers.  Specifically,  the  difference  in  guidance  changes 
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the  effort  of  each  available  strategy  and,  therefore,  provides  an  incentive  for  decision 
makers  to  use  different  strategies. 

A  number  of  studies  suggest  that  differences  in  information  displays  influence  strategy 
selection  through  changes  in  either  the  effort  or  the  accuracy  with  which  various  informa¬ 
tion  processing  activities  can  be  accomplished.  Specifically,  differences  in  displays,  task 
features,  and  decision  maker  knowledge  change  the  anticipated  effort  and  accuracy  of  each 
available  strategy  and,  therefore,  provide  a  cognitive  incentive  for  decision  makers  to  use 
different  strategies.  For  instance,  multi-attribute  weighting  decision  strategy  requires 
many  extractions  of  numerical  values.  Thus,  implementing  an  interface  in  a  way  that 
makes  numerical  value  extraction  easier  would  decrease  the  effort  required  for  using 
this  decision  strategy. 

Johnson  and  Payne  [19]  proposed  a  method  for  measuring  the  ‘effort’  associated  with 
decision  strategies  by  decomposing  strategies  into  a  sequence  of  component  processes, 
called  elementary  information  processes.  These  components  are  basic  cognitive  opera¬ 
tions  thought  to  be  common  to  a  wide  variety  of  tasks.  Examples  include  reading  an  item 
of  information  into  short-term  memory,  adding  two  numerical  items  together,  and  compar¬ 
ing  two  items.  A  measure  of  effort  can  be  calculated  from  the  total  number  of  component 
operations  required  to  execute  a  particular  strategy  in  a  particular  task. 

In  this  study,  some  strategies  for  choice  require  more  computational  effort  (weighted 
strategy)  while  the  others  require  little,  if  any,  computing.  Thus,  changing  to  an  interface 
that  makes  the  computations  of  the  weighting  technique  easier  would  decrease  the  effort 
required  for  the  more  demanding  technique  and  increase  the  likelihood  of  its  use. 

2.1.  Subjects 

The  46  subjects  were  part-time  MBA  students  at  two  major  Midwestern  universities. 
None  of  the  subjects  had  previously  been  subjects  in  any  DSS  experiment.  At  a  minimum, 
all  subjects  had  completed  an  introductory  course  in  Management  Information  Systems.  It 
has  been  demonstrated  elsewhere  that  student  subjects  can  serve  as  surrogates  for 
managers  in  experiments  on  decision  making  [20,21]. 

We  desired  novice  users  since  decisional  guidance  in  a  DSS  is  to  facilitate  novice  users, 
not  technical  experts  [13,22],  In  addition,  subjects  with  high  level  of  forecasting  knowl¬ 
edge  might  favor  a  certain  model  and  therefore  bias  the  results  [23].  Therefore  we  prequa¬ 
lified  the  subjects  by  asking  them  six  questions  about  forecasting  models  (Appendix  C)  to 
measure  their  knowledge  in  the  field.  None  of  the  subjects  answered  more  than  50%  of  the 
six  questions  correctly.  Also,  a  self-report  on  subject’s  knowledge  of  forecasting  models 
was  asked  as  a  check.  Only  three  subjects  self-reported  somewhat  good  knowledge,  but  all 
three  of  them  correctly  answered  only  two  out  of  six  questions  on  the  subject.  Therefore, 
all  of  the  subjects  were  considered  as  novice  users  included  in  the  study. 

2.2.  Design  and  procedure 

Two  experimental  sessions  were  conducted  in  this  study.  The  first  experimental  session 
was  a  within- subject  design  in  which  30  subjects  were  presented  each  different  guidance 
of  the  same  problem.  One  advantage  of  the  within- subject  design  is  statistical  power, 
which  can  help  to  compensate  for  the  limited  number  of  subjects  in  a  study  [24].  One 
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disadvantage  is  the  possible  introduction  of  order  effects  due  to  having  a  knowledge  of  the 
problem  from  one  trial  to  the  next.  The  second  experimental  session  was  a  sample  of  16 
using  the  same  procedure  except  for  segmenting  the  sample  population  into  two  groups  of 
eight  subjects.  Each  group  considered  only  one  of  the  two  guidances. 

The  subjects  were  to  solve  a  decision  problem  that  required  selection  of  an  appropriate 
forecasting  model  for  a  provided  scenario.  The  forecasting  models  for  selection  included 
those  in  Figs.  1  and  2.  The  criteria  used  for  selecting  a  forecasting  model  were  suggested 
by  [25].  The  scenario  is  in  Appendix  A.  Subjects  selected  one  of  four  decision  strategies, 
described  in  Appendix  B,  to  arrive  at  a  solution.  The  computer-based  participative-sugges¬ 
tive  guidance  was  developed  in  Lotus- 1-2-3  with  the  resulting  interface  being  those  of 
Figs.  1  and  2. 

Subjects  ‘self-recorded’  their  answers  using  paper  and  pencils.  They  were  asked  to 
select  a  forecasting  model  based  upon  the  information  given  in  the  scenario.  They  were 
also  asked  to  use  the  four  strategies  introduced.  The  ‘other’  category  is  combinations  of  the 
four  different  strategies  (e.g.  use  both  ‘comparing  by  pairs’  and  ‘satisficing’  strategies). 

At  the  start  of  the  study,  a  15-min  training  session  was  conducted  to  provide  subjects 
with  an  understanding  of  each  supported  selection  strategy.  Subjects  were  asked  to  use 
each  of  the  strategies  in  a  learning  exercise  to  select  an  apartment.  Any  question  about  how 
to  apply  the  decision  strategy  on  the  exercise  problem  was  answered  by  the  facilitator.  This 
ensured  the  subjects  understood  the  nature  of  the  experimental  task  and  the  decision 
strategies. 

In  the  first  experimental  session,  and  for  one  half  of  the  subjects  in  the  second  experi¬ 
mental  session,  all  subjects  were  asked  to  use  the  information  provided  in  Fig.  1  to  select  a 
forecasting  model.  They  were  allowed  use  of  only  paper  and  pencil  for  this  first  task. 
Subjects  recorded  their  decisions,  the  confidence  they  had  in  their  decision  (from  a  low 
confidence  of  1  to  high  confidence  of  100),  and  the  decision  strategy  they  adopted  for 
making  their  decisions.  A  total  of  20  min  was  allowed  to  complete  this  task. 

Following  completion  of  the  task,  each  subject  in  the  first  experimental  session  was 
asked  to  repeat  the  process  with  the  only  change  being  the  provision  of  spaces  for  using  the 
multi-attribute  weighting  method  on  the  computer  as  shown  in  Fig.  2.  Likewise,  the 
remaining  half  of  the  subjects  in  the  second  experimental  session  used  this  version  of 
the  spreadsheet.  This  change  in  software  represents  a  shift  to  participative-suggestive 
guidance  [13].  Again,  subjects  recorded  their  decisions,  the  confidence  in  their  decision, 
and  their  decision  strategy.  In  the  first  experimental  session,  after  completion  of  the  task, 
two  other  measures — perceived  ease  of  use  and  perceived  usefulness — were  elicited 
using  the  scales  of  [26].  These  scales  range  from  one  to  seven  with  seven  being  the 
favorable  end. 


3.  Results 

With  predefined  information,  only  ten  subjects  used  the  multi-attribute  weighting 
method  as  their  decision  strategy,  as  shown  in  Table  1 .  However,  with  the  multi-attribute 
weighting  method  having  a  participative-suggestive  guidance  interface,  35  subjects  chose 
that  particular  method  for  their  decision  strategy.  Fig.  3  shows  the  change  in  selection 
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Fig.  3.  Differences  in  method  selected  by  guidance  format. 

strategies  with  a  corresponding  change  in  guidance.  This  is  a  significant  change  in  propor¬ 
tion  according  to  Fisher’s  Exact  Test.  In  fact,  each  method  showed  a  significant  change  in 
proportions  indicating  that  the  software  did  influence  the  selection  of  the  model  as  we 
expected.  Also,  by  categorizing  the  results  into  the  Multi-attribute  Weighting  method  and 
all  others,  Fisher’s  Exact  Test  shows  no  difference  in  proportion  between  experimental 
sessions  for  either  guidance  method,  indicating  no  order  effects  were  significant  in  the  first 
experimental  session. 

Table  2  shows  the  decision  changes  among  the  28  subjects  in  the  first  experimental 
session  who  used  the  multi-attribute  weighting  method.  Among  the  eight  subjects  who  first 
used  the  method,  only  one  subject  changed  his/her  decision  and  the  other  seven  subjects 
stayed  with  the  same  decision.  However,  nine  of  twenty  subjects  who  did  not  adopt  the 
multi-attribute  weighting  method  as  their  first  decision  strategy  reached  different  decisions 
after  using  the  revised  format.  Eleven  subjects  did  not.  The  table  indicates  how  many 
subjects  actually  arrived  at  different  solutions  because  of  the  change  in  solution  procedures 
encouraged  by  participative- suggestive  guidance. 

Researchers  have  shown  a  strong  relationship  between  ease  of  use  and  usefulness  and  a 
user’s  acceptance  of  a  system  [26].  Table  3  shows  that  twenty-seven  out  of  the  thirty 
subjects  in  the  first  experimental  session  perceived  the  guided  decision  aid  as  easy  to  use 
and  twenty-four  out  of  thirty  subjects  perceived  the  revised  aid  as  useful  for  their  decision 
making  task.  The  non-parametric  Sign  Test  was  conducted  on  each  of  these  measures  [27]. 
The  neutral  response  of  four  on  the  scales  is  used  as  the  effective  zero.  Both  ease  of  use  and 
usefulness  were  in  the  favorable  range  at  a  significance  level  of  0.05,  from  which  we 
conclude  that  subjects  perceived  the  system  positively. 


Table  2 

Decision  changes  by  decision  strategy 


Multi-attribute  weighting 

Other  methods 

No  decision  change 

7 

11 

Decision  change 

1 

9 

Total 

8 

20 
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Table  3 


System  acceptance  metrics 

Ease  of  use 

Usefulness 

<  5a  rating 

3  subjects 

6  subjects 

>  =  5  rating 

27  subjects 

24  subjects 

a  The  scale  ra 

nged  from  1  to  7  with  7  being  i 

nost  favorable. 

4.  Discussion 

This  experiment  highlights  the  point  that  the  decisional  aid  of  a  DSS  can  significantly 
influence  users’  selection  of  a  decision  strategy  as  well  as  the  decision  outcomes.  It  is  not 
our  intent  to  argue  which  decision  strategies  should  be  implemented  for  the  problem  in  the 
experiment,  however,  DSS  interface  designers  need  to  take  the  amount  and  type  of 
guidance  into  account. 

The  type  of  guidance  can  impact  the  model  selected  dining  the  model  selection 
phase  of  decision  support.  This  may  be  a  two-edged  sword.  Indirectly  influencing  the 
model  selected  by  a  decision  maker  may  direct  the  user  into  the  selection  of  a  tech¬ 
nique  that  would  not  otherwise  be  chosen.  If  the  intent  of  the  DSS  is  to  allow  full 
flexibility  and  control  on  the  part  of  the  user,  then  the  resulting  impact  does  not  match 
the  intent.  However,  such  guidance  can  also  be  of  assistance  to  novice  users  unfamiliar 
with  the  problem  domain.  In  this  latter  case,  it  may  even  be  desirable  to  influence  the 
selection  of  a  particular  model  while  still  allowing  experienced  users  to  apply  their  own 
knowledge. 

An  advantage  of  this  type  of  guidance  is  that  it  can  be  effective,  but  also  subtle,  leaving 
the  users  the  perception  of  control.  This,  along  with  the  positive  perception  of  ease  of  use 
and  usefulness  can  lead  to  a  more  effective  use  of  the  system  [11,12],  A  disadvantage  is 
that  the  user’s  preferred  selection  strategy  may  be  overlooked.  Since  different  selections  of 
decision  models  do  indeed  result  in  different  solutions  [28],  the  DSS  may  inadvertently 
lead  to  a  less  preferred  solution. 

To  generalize,  the  guidance  portion  of  the  interface  design  cannot  be  taken  lightly.  DSS 
designers  traditionally  have  many  concerns  with  the  impact  that  their  design  may  have  on 
influencing  outcomes.  Ease  of  use  and  usefulness  are  appropriate  goals,  but  not  at  the 
expense  of  poor  solutions.  Thus,  structure  of  the  guidance  must  be  added  to  the  list  of 
design  concerns  that  can  bias  results.  The  list  already  includes  use  of  graphics  [2],  icons 
[29],  menus  [1],  feedback  [30]  and  other  dialogue  issues  as  well  as  the  many  issues  in 
model  inclusion  and  data  availability  [7,31]. 

The  study  results  must  be  viewed  with  caution  with  respect  to  generalizations.  The  only 
thing  shown  conclusively  is  that  a  variation  in  decisional  guidance  can  lead  to  very 
different  results  in  a  decision  setting.  The  formal  results  cannot  be  extended  to  multiple 
problem  domains  as  each  domain  may  be  unique.  These  limitations  do  not  lighten  the 
caution  a  DSS  designer  must  place  on  guidance  unless  s/he  has  explored  the  implications 
of  an  intended  interface. 

Further  exploration  of  guidance  may  help  to  clarify  existing  side  effects  and  allow 
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for  clearer  design  guidelines.  Can  participative-suggestive  or  predefined-informative 
guidance  make  a  DSS  more  effective  at  supporting,  not  influencing,  decision  makers? 
Can  suggestive  guidance  effects  be  limited  to  the  target  of  guidance  (structuring  vs. 
executing  the  decision-making  process)?  Does  decisional  guidance  affect  how 
much  time  the  decision  maker  spends  using  a  system?  Does  increased  ease  of  use 
translate  into  increased  frequency  of  use?  When  do  the  costs  of  learning  the 
guidance  mechanisms  exceed  the  benefits  of  using  them?  These  and  other  questions  are 
those  researchers  must  address  to  assist  in  the  development  of  future  Decision  Support 
Systems. 


Appendix  A.  Decision  scenario 

Imagine  that  you  are  a  computer  sales  manager  and  are  asked  by  top  management  to  use 
a  structured  technique  to  forecast  monthly  sales  in  the  future. 

•  The  project  is  to  be  done  in  three  months. 

•  You  and  your  staff  are  comfortable  with  mathematical  models. 

•  Historical  data  on  computer  sales  have  been  collected  industry  wide,  however,  only  the 
last  two  years  of  monthly  sales  data  are  available. 

•  There  is  US$100  000  available  for  this  project. 

•  A  fairly  detailed  monthly  sales  forecast  report  is  required. 

•  The  forecasting  model  will  need  to  be  updated  every  six  months. 

A  decision  support  system  has  been  proposed  by  a  marketing  analyst  to  support  the 
corporation’s  forecasting  needs.  Four  forecasting  models  are  suggested  by  the  analyst.  A 
forecasting  model  may  lead  to  a  satisfactory  prediction  or  an  unsatisfactory  prediction 
because  of  a  mismatch  between  the  characteristics  of  the  model  and  the  business  situation 
and  needs.  Your  job  for  now  is  to  select  the  most  appropriate  forecasting  model  for  your 
business  needs. 


Appendix  B.  Decision  strategies 


Multi-attribute  weighting:  This  technique  requires  that  you  consider  all  the  criteria, 
assign  them  weights  (in  terms  of  their  importance  or  value), 
and  then  combine  the  weighted  scores  to  choose  a  single 
model. 

Elimination  of  aspects:  When  using  this  approach,  you  eliminate  the  models  that  do  not 
meet  one  or  more  of  your  criteria.  For  example,  if  you  do  not 
have  the  resources  to  construct  a  model  that  is  rated  6  or  7  in 
expense,  then  you  eliminate  those  models.  You  evaluate 
models  against  each  of  your  important  criteria,  one  at  a  time. 
Then  you  choose  the  model  that  is  left. 
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Satisficing:  You  seek  the  model  that  meets  a  minimum  value  for  most  of 

your  criteria.  You  apply  this  judgment  to  each  criterion.  You 
select  the  model  to  which  most  (or  all)  apply. 

Comparing  by  pair:  When  using  this  technique,  you  compare  two  models  at  a  time. 

You  eliminate  the  least  attractive  model.  Repeat  this  process 
until  there  is  only  one  model  left. 


Appendix  C.  Test  of  forecasting  model  knowledge 


1 .  Which  of  the  following  forecasting  models  can  be  developed  most  quickly? 

Moving  average 
Exponential  smoothing 
Times  series  extrapolation 
Box-Jenkins 

2.  Which  of  the  following  forecasting  models  can  be  developed  least  costly? 

Moving  average 
Exponential  smoothing 
Times  series  extrapolation 
Box-Jenkins 

3.  Which  of  the  following  forecasting  models  requires  the  minimal  quantitative 
capabilities? 

Moving  average 
Exponential  smoothing 
Times  series  extrapolation 
Box-Jenkins 

4.  Which  of  the  following  forecasting  models  can  most  accurately  forecast  the  future 
under  significant  environmental  change?? 

Moving  average 
Exponential  smoothing 
Times  series  extrapolation 
Box-Jenkins 

5.  Which  of  the  following  forecasting  models  can  produce  the  most  detailed  results? 

Moving  average 
Exponential  smoothing 
Times  series  extrapolation 
Box-Jenkins 

6.  Which  of  the  following  forecasting  models  can  be  systematically  updated  most  easily? 
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Moving  average 
Exponential  smoothing 
Times  series  extrapolation 
Box-Jenkins 

Please  rate  your  knowledge  about  forecasting  models  from  1  (know  extremely  little)  to  7 

(very  knowledgeable). 

References 

[1]  B.  Mehlenbacher,  T.M.  Duffy,  J.  Palmer,  Finding  information  on  a  menu:  linking  menu  organization  to  the 
user’s  goals,  Human-Computer  Interaction  4  (3)  (1989)  231-251. 

[2]  M.G.  Sobol,  G.  Klein,  New  graphics  as  computerized  displays  for  human  information  processing  ,  IEEE 
Transactions  on  Systems  Man  and  Cybernetics  V19/4  (1989)  893-898. 

[3]  I.  Benbasat,  A.S.  Dexter,  P.  Todd,  The  influence  of  color  and  graphical  information  presentation  in  a 
managerial  decision  simulation,  Human-Computer  Interaction  2  (1986)  65-92. 

[4]  H.  Dunsmore,  Designing  an  interactive  facility  for  non-programmers,  Proceedings  of  ACM  80,  Nashville, 
TN,  October  27-29,1980,  pp.  475-483. 

[5]  P.  Todd,  I.  Benbasat,  The  use  of  information  in  decision  making:  an  experimental  investigation  of  the 
impact  of  computer-based  decision  aids,  MIS  Quarterly  16  (3)  (1992)  373-393. 

[6]  P.J.  Krause,  P.J.  Byres,  S.  Hajnal,  Formal  specification  and  decision  support,  Decision  Support  Systems  12 
(3)  (1994)  189-197. 

[7]  J.H.  Gerlach,  F.  Kuo,  Understanding  human-computer  interaction  for  information  systems  design,  MIS 
Quarterly  15  (1991)  527-548. 

[8]  J.M.  Carroll,  S.A.  Mazur,  LisaLeaming,  IEEE  Computer  19  (11)  (1986)  35-49. 

[9]  B.  Gaines,  The  technology  of  interaction-dialogue  programming  rules.  International  Journal  of  Man- 
Machine  Studies  14  (1)  (1981)  133-150. 

[10]  R.C.  Houghton,  Online  help  systems:  a  conspectus,  Communications  of  the  ACM  27  (2)  (1983)  126-133. 

[11]  M.D.  Good,  J.A.  Whiteside,  D.R.  Wixon,  S .J.  Jones,  Building  a  user-driven  interface,  Communications  of 
the  ACM  27  (1984)  1032-1043. 

[12]  J.M.  Carrol,  A.P.  Aaronson,  Learning  by  doing  with  simulated  intelligent  help,  Communications  of  the 
ACM  31  (1988)  1064-1079. 

[13]  M.  Silver,  Decisional  guidance  for  computer-based  decision  support,  MIS  Quarterly  1  (1991)  105-122. 

[14]  P.  Kleindorfer,  H.  Kunreuther,  P.  Schoemaker,  Decision  Science:  An  Integrative  Perspective,  Cambridge 
University  Press,  Cambridge,  England,  1991. 

[15]  J.B.  Black,  J.S.  Bechtold,  M.  Mitrain,  J.M.  Carrol,  One-line  tutorials:  what  kind  of  interface  leads  to  the 
most  effective  learning?  Proceedings  of  CHI’89  Human  Factors  in  Computing  Systems,  Austin,  TX,  1989, 
pp.  81-83. 

[16]  J.M.  Carrol,  J.  McKendree,  Interface  design  issues  for  advice-giving  expert  systems,  Communication  of  the 
ACM  30  (1987)  14-31. 

[17]  D.  Kahneman,  Attention  and  Effort,  Prentice-Hall,  Engelwood  Cliffs,  NJ,  1973. 

[18]  J.E.  Russo,  B.A.  Dosher,  Cognitive  processes  in  binary  choice,  Journal  of  Experimental  Psychology: 
Learning,  Memory,  and  Cognition  9  (1983)  676-696. 

[19]  E.J.  Johnson,  J.W.  Payne,  Effort  and  accuracy  in  choice,  Management  Sciences  30  (1985)  395-414. 

[20]  L.  Berkowitz,  E.  Donnestein,  External  validity  is  more  than  skin  deep:  some  answers  to  criticisms  of 
laboratory  experiments,  American  Psychologist  37  (3)  (1982)  245-257. 

[21]  W.E.  Remus,  Graduate  students  as  surrogates  for  managers  in  experiments  on  business  decision  making. 
Journal  of  Business  Research  14  (1986)  19-25. 

[22]  M.  Silver,  Descriptive  analysis  for  computer-based  decision  support,  Operations  Research  36  (1988)  904- 
916. 

[23]  G.J.  Udo,  J.S.  Davis,  Factors  affecting  decision  support  system  benefits.  Information  and  Management  23 
(1992)  359-371. 


J.J.  Jiang,  G.  Klein  /  Interacting  with  Computers  12  (2000)  469-481  481 

[24]  D.N.  Kleinmuntz,  D.A.  Schkade,  Cognitive  processes  and  information  displays  in  computer-supported 
decision  making:  implications  for  research,  Working  Paper,  The  University  of  Chicago,  Graduate  School 
of  Business,  Center  for  Decision  Research,  January  1990. 

[25]  P.M.  Georgoff,  R.G.  Murdick,  Manager’s  guide  to  forecasting,  Harvard  Business  Review  86  (1)  (1986) 
110-120. 

[26]  F.D.  Davis,  Perceived  usefulness  perceived  easy  of  use,  and  user  acceptance  of  information  technology, 
MIS  Quarterly  13  (1989)  319-339. 

[27]  M.G.  Sobol,  M.K.  Starr,  Statistics  for  Business  and  Economics,  McGraw  Hill,  New  York,  1983. 

[28]  G.  Klein,  H.  Moskowitz,  A.  Ravindran,  A  comparative  study  of  prior  vs.  progressive  articulation  of 
preference  approaches  for  bicriterion  problems.  Naval  Logistics  Research  Quarterly  33  (1986)  309-323. 

[29]  M.M.  Blattner,  D.A.  Sumikawa,  R.M.  Greenber,  Earcons  and  icons:  their  structure  and  common  design 
principles,  Human-Computer  Interaction  4  (1)  (1989)  149-177. 

[30]  R.M.  Hogarth,  C.R.M.  McKenzie,  B  J.  Gibbs,  M.A.  Marquis,  Learning  from  feedback:  exactingness  and 
incentives.  Journal  of  Experimental  Psychology  17/4  (1991)  734-752. 

[31]  S.  Rathnam,  M.V.  Mannino,  Tools  for  building  the  human-computer  interface  of  a  decision  support  system, 
Decision  Support  Systems  13  (1)  (1995)  35-59. 


Dr.  James  J.  Jiang  holds  the  Max  Watson  Professorship  of  Computer  Information  Systems  at  Louisana  Tech 
University,  Ruston,  LA.  He  obtained  his  PhD  in  Information  Systems  at  the  University  of  Cincinnati  in  1992. 
His  current  research  interests  include  software  project  management  and  IT  infrastructure  of  knowledge- 
based  organizations.  He  has  published  over  50  academic  articles  in  these  areas  in  the  journals  such  as  IEEE 
Transactions  on  Systems,  Men,  and  Cybernetics,  IEEE  Transactions  on  Engineering  Management,  Decision 
Support  Systems,  Communications  of  ACM,  and  Information  and  Management.  He  is  a  member  of  IEEE, 
ACM,  DSI,  and  AIS. 


Dr.  Gary  Klein  is  the  Couger  Professor  of  Information  Systems  at  the  University  of  Colorado  in  Colorado 
Springs.  He  obtained  his  PhD  in  Management  Science  at  Purdue  University  (Go  Boilers!).  Before  that  time, 
he  served  with  Arthur  Anderson  &  Company  in  Kansas  City  and  was  director  of  the  Information  Systems 
department  for  a  regional  financial  institution.  He  was  previously  on  the  faculty  at  the  University  of  Arizona, 
Southern  Methodist  University  and  Louisiana  Tech  University  and  served  as  Dean  of  the  School  of  Business  at 
the  University  of  Texas  of  the  Permian  Basin.  His  specialties  include  information  system  development  and 
mathematical  modeling  with  over  60  academic  publications  in  these  areas. 


